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METHOD AND APPARATUS FOR ELECTRONIC EQUALIZATION IN 
OPTICAL COMMUNICATION SYSTEMS 

Field of the Invention 

5 The present invention relates generally to signal equalization techniques, 

and more particularly, to methods and apparatus for equalizing optical signals. 



Background of the Invention 

The explosive growth of digital communications technology has resulted in 

10 an ever-increasing demand for bandwidth for communicating digital information, such as 
data, audio and/or video information. To keep pace with the increasing bandwidth 
demands, optical communication systems, with their large inherent channel capacities, 
frequently provide the backbone of modern communication systems. It is believed that the 
growth of fiber optic communications will continue for metropolitan and long-haul 

15 networks and that fiber optic communications will ultimately be applied even for access 
and local area (LAN) networks. In this on-going evolution of fiber optic communication, 
adaptive electronic equalization for combating impairments in fiber optic communication 
may play an important role in the following themes: 

Adaptive electronic equalizers for impairment compensation in fiber optic 

20 networks have been frequently studied. Initially, the dominant noise was quantum, shot or 
electronic thermal noise, which can be modeled effectively as additive Gaussian noise. 
Since the advent of efficient and low-noise fiber amplifiers in 1987, optical amplifiers 
have been used extensively to increase the transmission distance without conversion 
between the optical and electrical domains. A number of studies have explored a variety 

25 of equalizer structures for adaptive optical channel impairment compensation (ranging 
from feed forward type equalizers to maximum-likelihood estimators). More recently, 
interest is more focused on adaptive polarization mode dispersion (PMD) compensation 
since, in the wide-spread deployment of lOGbps optical equipment, substantial 
unpredictable PMD is accumulated over a long distance of legacy fibers, enough to cause 

30 network outage, though polarization mode dispersion compensation still remains an open 
topic. A need therefore exists for improved techniques for compensating for polarization 
mode dispersion and other noise in optical receivers. 
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Summary of the Invention 

Generally, a method and apparatus are provided for performing electronic 
equalization in optical communication systems. Optical noise in an optical receiver has a 
5 non-Gaussian nature. The present invention updates coefficient values in equalizers, such 
as feed forward equalizers or decision feedback equalizers, using higher-order algorithms 
in the Least-Mean-2Nth-Order family. In one embodiment, an optical receiver includes a 
photo-detector for converting a received optical signal to an electrical signal; and an 
equalizer for removing intersymbol interference from the electrical signal, the equalizer 

10 has a plurality of coefficients that are configured to be updated based upon a least-mean 
2N th -order (LMN) algorithm, where N is greater than one. 

The equalizer can be embodied as a finite impulse response filter that is 
configured to produce a first output signal responsive to the electrical signal. The first 
output signal is representative of a sum of associated electrical signals plus a weighted 

15 sum of previous electronic signals, where the previous signals are weighted by said 
coefficients. A slicer produces a predicted signal for each first output signal received from 
the finite impulse response filter; a subtractor produces an error signal proportional to the 
difference between the first output signal and corresponding ones of predicted signals and 
training signals; and a controller updates the coefficients responsive to the error signal. 

20 In a feed forward equalizer implementation, the controller updates a set of 

coefficients c(£ + l) at a time (k+1) as c(k) + PN[e(k)f N " x u(k) , wherein (3 is a preset 
step size, c(k) and e(£) are respective sets of coefficient and error signals at a time k, and 
\x(k) is an input signal at the time k. 

1 . In a decision feedback equalizer implementation, a first subtractor produces 

25 a second output signal that is a sum of one of the first output signals and a corresponding 
feedback signal; a slicer produces a predicted signal in response to each second output 
signal; a second subtractor produces an error signal representing a difference between the 
second output signal and a corresponding training signal or predicted signal; a feedback 
filter produces the feedback signal in response to corresponding ones of the predicted or 
30 training signals, the feedback signal being a weighted sum of the predicted or training 
signal, wherein weights in the sum being characteristics of the filter; and a controller 
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updates the set of weights in response to the error signal, wherein the controller is 
configured to perform the updates based upon a least-mean 2N th -order (LMN) algorithm 
where N is greater than one. The controller updates a set of the weights w(£ + 1) at a time 

(k+1) as w(£) + PN[e(k)] 2N ~ l r (k) , wherein fi is a preset step size, w(£) and e(£) are 
5 respective sets of weight and error signals at a time k, and r T (k) = [u(£),-a(A:)] , where 
u(£) is an input signal at the time k, and a(£) is a predicted or training signal at the time 
k. 

Another aspect of the invention addresses the non-symmetric nature of 
optical noise by employing adjustable slicer threshold control techniques to reduce bit 
10 errors (rather than employing a fixed threshold at the mid-point of two noiseless signal 
levels). A photo-detector converts an optical signal to an electrical signal; and a slicer 
produces a predicted signal in response to each input signal based upon a slicing threshold. 
The slicing threshold is varied based upon a signal distribution of the electrical signal. 

A more complete understanding of the present invention, as well as further 
15 features and advantages of the present invention, will be obtained by reference to the 
following detailed description and drawings. 

Brief Description of the Drawings 

FIG. 1 is a schematic block diagram of a conventional optical receiver; 
20 FIG. 2 is a schematic block diagram of a conventional digital optical 

receiver; 

FIG. 3 is a schematic block diagram of a conventional analog optical 

receiver; 

FIG. 4 is a schematic block diagram of a feed forward equalizer 
25 incorporating features of the present invention; 

FIG. 5 is a schematic block diagram of a decision feedback equalizer 
incorporating features of the present invention; 

FIG. 6 is a schematic block diagram of an alternate embodiment of the feed 
forward equalizer of FIG. 4 having adjustable slicer threshold control; and 
30 FIG. 7 is a schematic block diagram of an alternate embodiment of the 

decision feedback equalizer of FIG. 5 having adjustable slicer threshold control. 
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Detailed Description 

After the square-law detection in a photodiode of an optical receiver, the 
incoherent amplified spontaneous emission (ASE) noise induced in optical amplifiers 
5 becomes non-Gaussian, though a small amount of ASE can be effectively approximated as 
Gaussian. The present invention recognizes that the well-studied least-mean-square 
(LMS) algorithm, which is typically used to update equalizer coefficients, becomes 
suboptimal for optical noise in the sense that an optical signal-to-noise ratio (OSNR) 
penalty is observed even without the presence of PMD-induced intersymbol interference 

10 (ISI), though it is optimal for additive Gaussian noise. Various coefficient-dithering 
approaches were devised to replace LMS but generally suffer from slow adaptation rate 
and no guarantee of convergence. Typically, tap coefficients are adjusted in a trial-and- 
error manner with feedback from various types of error monitors, such as minimization of 
eye-monitoring pseudo-error count, maximization of eye-opening, Q-factor optimization, 

15 bit-pattern dependent threshold control, RF power maximization, or FEC-error-count 
minimization. 

Despite continued research in the area, an electronic equalizer for optical 
receivers has still not been implemented. First, since the bit rate in fiber optic 
communication systems is typically 10 Gbps or above, it is difficult to implement 

20 complicated algorithms, as opposed to the case of the comparatively low bit rates 
associated with wireless communications. In addition, the nonlinear nature of a fiber 
optic channel provides additional challenges. The nonlinear nature of a fiber optic 
channel is due to the quadratic detection of photons (electromagnetic waves in the ultra 
high frequency range) instead of electrical field detection in electronic wired or wireless 

25 communications. Electronic compensation deals with noise and ISI after quadratic 
detection, but most readily-used electronic compensation algorithms are based on a linear 
system and are thus less effective in such a fiber optic environment. 

The present invention recognizes that optical noise after the square-law 
optical to electrical conversion in an optical receiver will have a non-Gaussian nature. In 

30 addition, the present invention recognizes that higher-order statistic techniques in the 
least-mean 2N th -order (LMN) family are better suited for updating equalizer coefficients 
in the presence of non-Gaussian noise than the typical LMS algorithm. 
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Thus, one aspect of the present invention updates coefficient values in 
equalizers, such as FFE or DFE equalizers, using higher-order algorithms in the Least- 
Mean-2Nth-Order family. In addition, due to the non-symmetric nature of optical noise, 
another aspect of the invention employs adjustable slicer threshold control techniques 
5 based on an input signal distribution to minimize bit errors (rather than employing a fixed 
threshold at the mid-point of two noiseless signal levels). 

Optical Noise Background 

In a fiber optic link, a number of optical amplifiers, either erbium-doped 
fiber amplifiers (EDFA) or Raman amplifiers (RA), are typically employed to strengthen 

10 the optical signal, but at the same time add in the incoherent amplified spontaneous 
emission (ASE) noise (commonly called optical noise). In the quantum regime, the ASE 
noise follows the Bose-Eistein distribution and the optically amplified coherent light 
exhibits a noncentral-negative-binomial distribution. 

As previously indicated, after square-law detection using a photodiode in 

15 an optical receiver, the noise becomes non-Gaussian (even though a small amount of ASE 
can be approximated as effectively Gaussian). FIG. 1 is a schematic block diagram of a 
conventional optical receiver 100. As shown in FIG. 1, an optical receiver 100 tyipcally 
filters the received optical signal using an optical filter 1 10, before converting the optical 
signal to an electrical signal using a photo-detector (diode) 120. The electrical signal is 

20 then amplified by an amplifier 130, and again filtered by an electronic filter 140, prior to 
performing a clock/data recovery process 150. 

Precisely, the photocurrent I(t) induced in the photo-detector 120 is 
proportional to P N (t), the power of the noise-coupled optical signal E(t) + n 0 (t), where 
E(t)=y(t,L): 

25 /(0*^(0 = |V0®^ 0) 

Here E(t) = h Q {t)<& E(t),n Q (t) = A 0 (0® " 0 (0> and P{t) = \h 0 (t)® E(t)\ 2 is the power of 

the noiseless optical signal E(t) convolved with the optical filter 110. Therefore, the effect 
of the optical filter 110 on the optical noise is to transform white noise to colored noise 
whose autocorrelation function is no longer a S function: 

30 %(0«o0 2 )*} = 4^. -0;^o(0«oft)} = 4|AoO, -r)h 0 (t 2 -r)dt(2) 
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This photocurrent I(t) is then converted to a voltage signal by the transimpendance 
amplifier (TIA) 130, and further filtered by an electronic filter 140, H e (f)= 3 {he(t)}, to 
generate the input signal u(t) for a clock-data-recovery (CDR) 150 in a conventional 
optical communication system: 

5 u(t)ozh e (t)®P N (t) = h e (t)®P(t) + 2h e (t)®^^ (3) 

In addition to the noise filtering effect, h^t) has an effect similar to that of ho(t) on the 
noise, making the resultant noise more correlated in time. This electronic low-pass filter 
can be modeled as a 3rd-order or 4th-order Bessel filter of bandwidth ranging from 7 GHz 
to 13 GHz at 10 Gbps. As a result, the noisy input signal u(t) contains a signal-dependent 

10 non-symmetric Gaussian second term having a variance determined by the signal levels of 
bits 0/1 and a nonzero-mean noncentral % r third term. 

It should be noted that the mean of u(t) is nonzero and varies at different 
optical signal-to-noise ratio levels (OSNR). This noise property is different from 
common electronic thermal noise or even optical noise after the combination of an 

15 interferometer and a balanced receiver. The OSNR for a single polarization state is 
defined as the ratio of average signal power to noise power within a fixed bandwidth of 
O.lnm (or equivalently 12.5 GHz for the 1550 nm wavelength) in that polarization state 
before the optical filter 110: 



OSNR = 



j[|^(o| 2 P(i)+|^o(0 fp(0) 



dtIT 

— (4) 



a ASE *l2.5GH z 

20 where E 0 (t) is the noiseless optical field for bit 0 when the extinction ratio 

ER = J|£, (0 (dt I $\E 0 (t)fdt is finite, and p(l) or p(0) are the probabilities of bits 1 and 
0, respectively. 

Electronic PMD Compensators 
FIGS. 2 and 3 illustrate exemplary conventional architectures for digital 
25 and analog receivers, respectively. As shown in FIG. 2, the digital receiver architecture 
200 includes a variable gain amplifier 210, an anti-aliasing filter 220, an analog-to-digital 
converter 230, a clock recovery circuit 240, an adaptive equalizer 250, and an automatic 
gain control stage 260, that operate in a well known manner. Of interest here, the analog- 
to-digital converter 230 digitizes the incoming bandlimited analog signal for further 
30 processing in the digital adaptive equalizer. Though more complex functionality can be 
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achieved in low-power CMOS digital circuitry coupled with pipelined/parallel data flows 
after digitization, the ADC 230 is considered one of the bottlenecks for digital 
implementation of advanced equalizers. The current state-of-art ADC can provide about 
6-bit accuracy at 10 Gbps but it becomes challenging at 40 Gbps. Furthermore, the power 
5 consumption of the ADC 230 at such a high speed may offset the low-power benefit of 
digital CMOS circuits. 

As shown in FIG. 3, the analog receiver architecture 300 optionally 
includes a variable gain amplifier 310 and anti-aliasing filter 320 and includes a clock 
recovery circuit 340, an adaptive equalizer 350, and an automatic gain control stage 360, 

10 that operate in a well known manner. The analog receiver architecture 300, and 
particularly the simple analog tapped-delay-line equalizer, is an attractive alternative in 
the absence of ADC for providing a practical electronic PMDC (EPMDC) solution. 
Regardless of the adopted implementation (digital or analog), the operations of an 
equalizer can be described in the discrete-time Z-domain. Equalizer algorithms are 

15 expressed herein in the discrete-time domain without explicitly referring to a digital or 
analog implementation. 

Equalizer Architectures 

A number of electronic equalizer structures, such as feedforward equalizers 
(FFE), decision feedback equalizers (DFE), maximum likelihood sequence estimators 

20 (MLSE), and maximum a posteriori (MAP) equalizers, have been considered. 
MLSE/MAP typically have substantial complexity and compromises are often associated 
with a practical implementation. Specifically, MLSE/MAP implementations have 
exponential complexity 0(e N ), while FFE/DFE techniques require only linear complexity 
0(N). Nonetheless, FFE/DFE implementations offer adequate performance. 

25 FIGS. 4 and 5 are schematic block diagrams of a feed forward equalizer 

(FFE) 400 and a decision feedback equalizer (DFE) 500, respectively, incorporating 
features of the present invention. As shown in FIG. 4, the feed forward equalizer 400 
includes a finite impulse response (FIR) filter 410, a slicer 420, an adder 430 and a weight 
update unit 440. The N-tap FFE coefficients are {c_ L (k) 9 ... 9 c L (k)} for jV = 2L + 1 at 

30 t = kT s in the discrete-time domain. Here T s =l/f s is the sample period. It is noted that 
C(z)=Z{c(k)} and both C(z) and c(k) therefore refer to the FFE coefficients. If the 
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oversampling ratio, R (T/T s ) is greater than 1, then the FFE is called a fractionally-spaced 
equalizer. In vector form, these weights can be denoted as c r (k) = [c_ L (A;), . . . , c L {k)\ . The 

input data sequence is u T (k) = [u(k + L),. ..,u(k-L)]. Therefore, the slicer input, 
s(k) = x(k) = c T (k)u(k) . The weight updating unit 440 of C(z) (WUD{C}) adapts a set 
5 of coefficient values, c(k) , based on an adaptive algorithm. In conventional equalizers 
the least-mean-square (LMS) algorithm has been widely used. To make a hard decision, 
the slicer 420 determines the estimated symbol d(k) (or bit in the On-Off Keying (OOK) 
case) in reference to a slicer threshold s(k). The decision point (slicer threshold 9) is 
typically fixed at the mid-point of two noiseless signal levels. For the case of only two 

10 signal levels, the slicer is simply a high-speed comparator as used in a conventional clock- 
data-recovery (CDR). 

FIG. 5 is a schematic block diagram of a decision feedback equalizer 500 
incorporating features of the present invention. As shown in FIG, 5, the decision feedback 
equalizer 500 includes a finite impulse response filter 510, a slicer 520, an adder 530, a 

15 feedback equalizer 550, a weight update unit (WUD(C)) 540 and a weight update unit 
(WUD(B)) 560. Thus, the decision feedback equalizer 500 includes both an FFE 510 and 
a feedback equalizer (FBE) 550, where s(k) = x(k) - y{k) = c T (k)u(k) - b T (k)a(k) with 
the set of coefficients of the FBE filter defined as b T (k) = [b^(k),.,.,b M (k)]. Here 
a r (/r) = [a(k -1),- ■• ,a(k - M)] with a(k) = d(k) (desired symbol) in the training mode, 

20 but a(k) = d(k) (estimated symbol) in the normal operating mode. The weight updating 
units for FFE and FBE are denoted as WUD(C) and WUD(B) respectively and can employ 
a variety of adaptive algorithms as in the case of the FFE. 

Higher-Order Statistics - Least-Mean 2N th -Order (LMN) Algorithms 

It has been shown that the LMS algorithm is efficient under the assumption 

25 of temporal whiteness and Gaussian disturbances, providing theoretical justification for 
the excellent robustness of LMS. However, after square-law detection by a photodiode, 
optical noise becomes nonzero-mean signal-dependent^ 2 . As a result, the conventional 
LMS algorithm introduces an OSNR penalty even without PMD, although the LMS 
algorithm is optimal for additive Gaussian noise (e.g., thermal noise). Due to the non- 
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Gaussian nature of optical noise, the present invention employs the higher-order 
counterparts of LMS which are found to yield better results than LMS. 

Thus, according to the present invention, the weight updating unit 440 
(WUD{C}) in the FFE 400 adopts an algorithm in the LMN family with order N greater 
5 than one minimizing the cost function, J(k) - E{[e(k)] 2N } where the sheer error, e(k), 

equals a(k) - s(k). Here, the desired signal a(k) equals d k in the normal operating mode 
whereas a(k) equals d(k) in the training mode. At a time (k + 1), the algorithm uses the 
values of c(k), e(k) and u(k) from previous time k to determine updated coefficients 
c(k + \) as follows: 

10 c(* + l) = c(k)-£-Vc{[e(k)] 2N } = c(k) + /3N[e(k)] 2N - x u(k) (5) 

where J3 is the preset step size. For N equal to 1, the LMN algorithm becomes LMS. For 
N equal to 2, it is called the least-mean fourth-order (LMF) algorithm. The present 
invention employs algorithms where N is greater than one. Since LMS belongs to the 
LMN family, the extensive knowledge of LMS in both theory and implementation can 
1 5 also be applied to the LMN family. 

In a similar way, consider the DFE 500 shown in FIG. 5, where 

e(k) = a(k) - s(k) 9 and s(k) = x(k) - y(k) = c T (k)u(k) - b T (k)a(k) . Define 

w 7 (£) = [c(k)-b(k)] and r T (k) = [u(*),-a(£)] where 

c r (k) = [c, L (*)... c L (k)], u T (k) = [u(k + I)... u(k - L)l 
20 b T (k) = [b x (&)... b M (k)l a T (k) = [a(k -l)...a(k- M)] . The variable e(k) can be rewritten 

as a(k)-w T (k)r(k). Therefore, the set of coefficients of the FFE and FBE (feedback 

equalizer) are updated at a time (k + 1) as follows to minimize the cost function, 

J(k) = E{[e(k)] 2N }: 

w(* + 1) = w(/t) - ^ Vw{[e(k)] 2N } = w(t) + /3N[e{k)] 2N - x v(k) (6) 
25 The first and second moments (equivalently mean and variance) are 

sufficient to describe Gaussian noise distributions. As a result, it is not necessary to 
account for higher moments in the cost function and the second-order cost function in 
LMS is enough for Gaussian noise. For non-Gaussian noise, higher moments are required. 
Intuitively, the higher-order cost function wages a large penalty for a large sheer error to 
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account for higher moments and cause the noise distribution to be more compact for non- 
Gaussian noise. 

Slicer With Automatic Threshold Control (ATC-SLICER) 
In the uncompensated on-off keying case, the two noiseless signal levels 
5 for bits values 0 and 1 are normalized to 0 and 1 with the mid-point defined at 0.5 (or 
equivalently to {-1, 1} with the mid-point at 0). In view of the non-symmetric 
distributions for bits 0 and 1 (having different variances) reflected in the signal-dependent 
second term in Equation 1 after square-law detection, the present invention recognizes that 
the optimal threshold point is usually not the mid-point (i.e., is usually not the value of .5 
10 for the bit values 0 and 1). Therefore, another aspect of the invention provides an 
automatic threshold control (ATC) algorithm to track the incoming signal profile and 
automatically adjust the threshold in the CDR for a reduced bit error rate. 

In theory, an ATC algorithm can search an entire simulation sequence for 
the best threshold value to obtain the least number of bit errors. In a practical system, 
15 however, the ATC algorithm can accumulate signal distribution information within a 
window of finite duration for optimal threshold calculation to allow tracking of slowly 
varying non-stationary channels. When the signal distribution information is accumulated 
at repetitive time instants over a period of time, a "histogram" of the distribution is 
obtained. The histogram will reveal a "broadening" of the noise distribution for bit 1 in 
20 comparison to that for bit 0 and can be used to identify a slicer threshold that will provide 
a reduced bit error rate. 

In the equalizer structures illustrated in FIGS. 4 and 5, the decision point 
(slicer threshold, 9) is fixed at the mid-point of two noiseless signal levels. Similar to the 
uncompensated case, the slicer input distributions for bits 0 and 1 are also non-symmetric 
25 and thus result in the optimal threshold point being away from the midpoint in certain 
cases. 

According to another aspect of the present invention, shown in FIGS. 6 and 
7, alternative architectures for FFE/DFE are provided with adjustable threshold control in 
the slicer 620, 720 (ATC-SLICER). Other than the adjustable threshold control in the 
30 slicer 620, 720, the feed forward equalizer 600 and decision feedback equalizer 700 
operate in the same manner as discussed above in conjunction with FIGS. 4 and 5, 
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respectively. The function of the ATC-SLICER 620, 720 is to track the slicer input 
histogram and adjust the slicer threshold accordingly. 

After a cold startup of the weight-update unit 640, 740 utilizing blind 
equalization algorithms, such as reduced constellation algorithm (RCA) or constant 
5 modulus algorithm (CMA), the ATC-SLICER 620, 720 can start to build up a slicer input 
profile to determine the best threshold for the slicer, and then dynamically track and adjust 
the optimal slicer threshold. In a digital implementation, an ADC is needed to convert the 
10 Gbps signal to digital values and the number of digital values required must be 
determined through fixed-point analysis to minimize the impact of accuracy-induced 

10 performance degradation. After digitization, a highly pipelined and parallel data-flow 
architecture can be devised to allow real time processing of the digitized signal. 

In an analog implementation form, no ADC is needed, but a number of tap- 
delayed amplifiers with an analog adder form a finite impulse response (FIR) filter as the 
basis of FFE/FBE. The ATC-SLICER 620, 720 can be implemented as a high-speed 

15 comparator with the threshold controlled by an external device that collects the 
compensated signal distribution over time. A mixed-signal implementation is also 
possible by utilizing the high-speed analog FFE/FBE with low-speed digital WUD. 

Automatic threshold control techniques (ATC) typically process raw 
analog input from the transimpedance amplifier (TIA). For the adaptive equalization 

20 associated with an aspect of the present invention, the input to the ATC- ALICER control 
block is the slicer input s(k). In one of many possible digital implementations, a 
histogram is established in two arrays of memory, bini(i) and bin 0 (i), for i=0,...,B+l 
corresponding to the intervals {(-inf,-l),[-l,-Hd),[-l+d,-l+2d),...,[-l+(i-l)d,-l+id),...,[l- 
d,l],(l,inf)}, where d=2/B and B is the number of bins that form the range from -1 to 1. 

25 The value of B dictates the accuracy of the final threshold determination but a large B 
value requires more memory space. For example, B could be 128, 256 or 1024. If s(k) is 
in the interval of [-l+(i-l)d, -1+id), then the count in the memory bini(i) is incremented by 
one for the slicer output having a value of 1 and bino(i) is incremented for the slicer output 
having a value of 0. The bit error can be estimated as follows: 

30 • e(0)=bin,(0)-bin 0 (0), 

• e(i)=e(i-l)+bin,(i) - bin 0 (i), for i=l,...,B+l. 
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The optimal threshold is determined by finding the minimum of {e(i)}. To avoid the 
detrimental memory effect for a changing optical channel, a reset signal is sent 
periodically to clear the memory arrays and a histogram is rebuilt fresh in accordance with 
the above rule. 

5 It is to be understood that the embodiments and variations shown and 

described herein are merely illustrative of the principles of this invention and that various 
modifications may be implemented by those skilled in the art without departing from the 
scope and spirit of the invention. 



